Method and apparatus for concealing portions of a video screen

ABSTRACT

A simple, cost-effective, and robust method and system to obstruct crawls, logos, and other annoying and distracting images overlaid on a video signal and displayed on a TV set or monitor is provided. The method and system may detect the presence of the unwanted images and block them automatically, or they may accept manual input from the user via a handheld control device to block or obstruct these images.

RELATED APPLICATIONS

None

BACKGROUND OF THE INVENTION

A. Field of the Invention.

This invention relates to a method and system for selectively replacingportions of a displayed video screen. More particularly a system andmethod are provided in which extraneous or undesirable image elements ona screen are blocked or replaced with more acceptable image elements.

B. Description of the Prior Art.

TV channels, especially network channels, have lately taken tooverlaying various “additions” or “enhancement” onto a main videoscreen. A particularly annoying (to some people) example is “the crawl”,which is a horizontal stream of text that is overlaid on top of an imageand generally moves from right to left across the bottom of a screen.Other examples of potentially annoying extraneous image elements includepop-ups that advertise upcoming shows, sports-related “scoreboard” and“stats”, and logos added for branding purposes.

Other extraneous image elements include an extra border surrounding themain screen. On at least some TV sets it is not possible to remove thisborder from the screen, even using TV viewing modes such as zoom mode.Moreover, in most instances, even if the zoom mode removes most of anextraneous image element, some portions thereof, usually disposed alongthe sides and/or the top of the screen remain visible. While this may begood branding for the broadcaster or distributor of programs, it isquite distracting to the viewer, especially when the extraneous imageelements are very bright or have a light color and viewing scenesprovide dark backgrounds.

The extraneous image elements may, at one time or another, be perceivedas distracting and/or annoying by almost all the viewers. In fact, oneof the present inventors has observed a TV viewer construct a crudemanual solution to this problem: a piece of heavy paper taped over thebottom portion of a TV screen to cover up a “crawl” because it was toointrusive.

Thus, a need exists for selectively removing or replacing extraneousimage elements from a video screen. The removed elements can be replacedwith less visually annoying and distracting elements.

Although there have been attempts or claims to solve this problem, thereis no concrete, definitive solution. In U.S. patent application Ser. No.10/696,141 (Publication US 2005/0094032), the inventor proposes “toprevent the hideously scrolling ticker from being presented to theviewer”. The viewer accomplishes this “simply by manipulating a buttonon a remote control” to “suppress [the] scrolling part from view”.

Another attempt to eliminate the annoying and distracting overlays isdetailed in a paper entitled “Erasing Video Logos Based on ImageInpainting” (Proc. IEEE International Conference on Multimedia and Expo,Lausanne, August 2002). The authors of the paper describe methods ofdetecting video logos and “inpainting” (filling in of any gaps inphotographs) the picture areas from which they are removed. The paperdescribes manually selecting the rough rectangular region of the videoclip that encloses the logo. Then the color frames of the selectedregion are transformed into grey-scale frames, and the contrast in thegrey-scale video is enhanced. The best-quality logo frame is alsoobtained from the video clip. After the logo is obtained, the regionwith logo is restored by the image “inpainting” technique (i.e., thelogos are erased from the video frames). However, the method describedin the paper employs an algorithm, which, although less complicated thanthose published previously, is still compute-intensive. For the goal ofreplacing a logo, crawl, or other offending video with less offensivevideo, the method and system of the present invention provide a simplerand more cost effective solution.

SUMMARY OF THE INVENTION

The present invention meets the current need for a superior means toobstruct or remove the annoying and distracting image elements of abroadcast program.

The invention uses a simple, inexpensive and robust method and system tocover up, or more appropriately, replace undesirable image elements. Thepresent invention could be provided as a premium service, possibly apaid service from cable companies, satellite companies or otherbroadcast content providers to enable any given subscriber to eliminatecrawls or logos on a screen. These undesirable image elements arereplaced by blocks having suitable sizes and colors. In one embodiment,the sizes and colors of the blocks are either preselected or customizedby the viewer and the colors are preselected, or selected by the viewerfrom a pallet. In another embodiment, the colors of the blocks aredynamically changed to match features of a current image thereby makingthe blocks less visible.

BRIEF DESCRIPTION OF THE DRAWINGS

The features and advantages of the present invention will become furtherunderstood with reference to the following description, appended claimsand accompanying drawings, in which:

FIG. 1 is an illustration of a video media player in a so-called “fullscreen mode” with an undesirable border around the perimeter of thescreen;

FIG. 2 a is an illustration of a typical video screen with a text crawlat the bottom of the screen;

FIG. 2 b is an illustration of the typical video screen of FIG. 2 a withthe crawl at the bottom of the screen blocked out;

FIG. 3 a is an illustration of a typical video screen with a border anda logo around the outer portion of the screen;

FIG. 3 b is an illustration of the typical video screen of FIG. 3 a withthe border and logo blocked by “rounding off” corners;

FIG. 4 a is an illustration of a typical video screen with underlyingborders and/or logos blacked out;

FIG. 4 b is an illustration of the typical video screen of FIG. 4 awhere the underlying borders and/or logos are obscured by overlayingthem with different image elements having colors similar to theadjoining portions of the broadcast image;

FIG. 5 a is an illustration of a typical video screen with a scoreboardat the top left corner of the screen;

FIG. 5 b is an illustration of the typical video screen of FIG. 5 a withthe scoreboard blocked by a video of a similar color to the video image;

FIG. 6 shows a block diagram constructed in accordance with the presentinvention to implement the embodiments shown in the Figures above;

FIG. 7 shows a flow chart for a method of identifying the predominantcolor of a region and/or identifying a region of a video screen as atwo-color region;

FIG. 8 shows a flow chart for a method of detecting the presence of anopaque logo in a video;

FIG. 9 shows a flow chart for a method of detecting the presence of asemi-transparent logo in a video.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

In one preferred embodiment, the invention allows the viewer (and/orothers, e.g. potentially, a service provider) to replace certainportions of a displayed broadcast program in a static manner. The term“broadcast program” is used herein to include any content from a serviceprovider that is distributed to many viewers, said content includingsome image elements that may be undesirable or objectionable to at leastsome of the viewers. The broadcast program can be distributed over theairways as standard TV signals, over cable connections, satellitetransmissions, Internet streaming and other similar means. FIG. 1provides an illustration of a typical broadcast program with undesirableimage elements. In this example, generated by AOL's Hi-Q Video Player,the display 10 is provided with a border 12. The border 12 contains abottom segment including an advertising link 14, a running time bar 16,and a “Compact Size” button 18, and other control buttons 19 all ofwhich may be undesirable. The top segment may include a title bar andother information or active controls. In addition, the border 12 itselfis white color, which may annoy some viewers because the border may bemuch brighter than the picture content. The present invention allows theviewer to block or replace the border and the unnecessary extra featuresor image elements of the border 12, as described in more detail below.

FIG. 2 a illustrates a broadcast program with a crawl 20. The programcan be displayed on a TV screen or PC monitor with the crawl 20 beingdisposed at the bottom of screen as shown and normally consisting ofstreaming text. As illustrated in FIG. 2 b, in the present invention theundesirable image element (e.g., the crawl 20) is replaced by a staticblack bar or block 40 at least while the crawl is present. In oneexample of this embodiment, a viewer can simply push a single “blockcrawl” button on a remote control (discussed more fully below) resultingin a black bar 40. The term “Remote control” in the context of thisinvention shall mean any wired or wireless device, preferably a handhelddevice, which has the ability to control the system implementing thepresent invention. The crawl height and location tend to beapproximately the same for all channels therefore this embodiment can beimplemented using the same black block overlaid on the bottom of the TVscreen for all the channels.

FIG. 6 shows an apparatus for implementing the invention. In thisfigure, a receiver 602 receives a broadcast program and feeds it to ablock overlay device 604. The output of the device 604 is then fed to ascreen 605 for display to one or more viewers. The operation of theapparatus, including the device 604 is responsive to commands from aremote control 606. The elements shown in FIG. 6 (with the possibleexception of the remote control 606) may all be incorporated into a TVset or a set top box or other similar devices. Alternatively, parts ofthe apparatus 600 may be provided as a separate standalone system in itsown housing (not shown). For example all of the parts of the apparatus600 except 602, 605, 608 and 610 may be provided as a standalone systemthat, e.g., accepts a “Video out” signal output (not shown) from part602 of a TV set comprising parts 602 and 605, and returns processedvideo via a “Video in” input (not shown) to TV set part 605. Such astandalone system may also accept commands from remote control parts 608and 610, where the remote control may control both the TV set and thestandalone system. Alternatively, all of the parts of the apparatusexcept 602, 605, 608, and 610 may be provided as a standalone systemthat, e.g., accepts an “RF in” signal from a set top box (STB),demodulates the RF signal to baseband, processes the baseband videosignal, remodulates the baseband signal to RF, and provides an “RF out”signal to a TV set comprising parts 602 and 605.

For the embodiment of FIGS. 2 a, 2 b, a user can selectively activate ordeactivate the block overlay device 604 by using remote control 606. Forexample, the remote control 606 may have a pushbutton 608 to controldevice 604. When device 604 is deactivated, the images from receiver 602are passed through to screen 606, and include the extraneous crawl 20.When the device 604 is activated, the block overlay device generates ablack block 40 that is overlaid on top of the images from 602 resultingin the images, such as the one in FIG. 2 b.

In a somewhat more sophisticated example of this embodiment, the remotecontrol 606 may feature two buttons 608, 610 for providing a “blockcrawl” and a “round corners” function respectively. This may produce arelatively pleasing “rounded off” TV picture that can eliminate suchfeatures as logos, sports scoreboards, and even “picture outsidepicture” ads that are sometimes used during events that have noacceptable interruption points, such as soccer games. These concepts aredemonstrated in FIGS. 3 a and 3 b.

FIG. 3 a illustrates a video monitor or TV screen with a border aroundthe video image, and a logo in the lower right corner of the videoimage. The border 30 includes several image elements including sidesegments 50 disposed around the main screen area 60, at least one cornerelement 70 with a logo or other information and two segments 80 disposedat the top and the bottom of the screen.

According to the present invention, the recited image elements arereplaced by blocks forming a black frame 90. The frame 90 is generatedin response to the activation of the first button 608 on the remotecontrol 606. In addition, the corner elements 70 are replaced by blackcorner blocks 85 when the corner (or rounded off) button 610 isactivated on the remote 606, thereby replacing the corners of the image.In one embodiment shown in FIG. 3 b, the corners 85 are triangular. Inan alternate embodiment, the hypotenuses of the triangles are replacedwith curved edges (not shown). FIG. 3 b illustrates the images that aregenerated by the block overlay device 604 with a black border that isused with the rounded corners, or corners replaced by triangular-shapedblack portions, to replace the logo and the annoying and distractingborder segments.

The viewer can select the less-obtrusive method of rounding off thecorners based on the viewers preference (i.e., using a curved-line,triangular or other standard or custom shapes pleasing to the viewer).Again, in this example, a default-sized, rounded off picture may proveacceptable given that TV broadcasters have a fairly good understandingof how much of the picture they can overlay without generating excessivehostility from viewers (i.e., the maximum sizes of items that areoverlaid on the corners of images can be predicted with a high degree ofcertainty)

In another embodiment, the viewer is provided with a more sophisticateduser interface that allows him or her to replace arbitrarily sized andshaped picture regions. For example, the user interface may allow theviewer to replace, with black-colored rectangles, one to four (or more)arbitrarily-sized rectangular picture regions on each side of the TVpicture. In this example, the viewer is provided with the ability toblack out certain characters or scenery at any of the top, bottom, leftor right side of a TV picture if it is not to the viewer's liking forany reason, and not just to remove logos, crawls and other unnecessaryoverlays. The viewer may simply want to black out a certain character orbackground on one side of a TV picture, for example, and this embodimentgives the viewer this ability. Thus, with reference to FIG. 3 a, theviewer may select two rectangular black bars to replace top and bottomborder segments and a black rectangle to replace the logo. The blackbars are of a pre-selected size, and a number of blocks of differentsizes may be made available to the viewer. In another embodiment of theinvention, the viewer is given the capability of resizing the black barsas necessary to accommodate the size of the broadcast picture elementthat the viewer wants to replace.

These concepts may be implemented as follows. Referring to FIG. 6, thedevice 600 includes a block generator 612, a block selector 614 and ablock shape library 616. Either at the beginning, or during thepresentation of the broadcast program, the remote control 606 is used bya viewer to select one or more blocks through block selector 614. Theblock selector 614 selects the blocks from a block library 616. Forexample, the library 616 stores blocks of several different shapes,including horizontal bars, vertical bars that can be placed along theedges of an image, as well as triangles or other shapes suitable for thecorners. The sizes of the block shapes may be fixed, or the viewer maychange their sizes before they are fed to the block selector 614. Theblock selector 614 then sends the characteristics of the blocks to theblock generator 612 provides the block(s) to device 604 forsuperposition or overlay onto the image.

In another embodiment of the invention, the user interface supportsdrawing on the TV screen 605 with the remote control 606 or anotherinput device, so that the viewer can select one of many preset shapesfrom library 616, draw an arbitrary shape, or resize the selected ordrawn shape as required. The block is then superimposed on the screen605 by the block drawing device 618. The viewer can also move the blockto any desired position on the screen 605 with device 618. Once theviewer is satisfied with the size, shape and position of the block, herequests this information to be loaded into the block selector 614 fortransmitting to device 604.

In the embodiments described above, the undesirable or extraneous imageelements are replaced with black blocks of various shapes and sizes. Inanother embodiment of the invention, instead of black blocks, the imageelements from a broadcast program are replaced with blocks of one ormore sizes and one or more colors selected from an arbitrary palette(i.e., the replacement blocks are “filled” with any color selected fromthe palette provided to the viewer). More particularly, device 600 isoptionally provided with a color selector 620. The viewer can thendesignated one or more colors for the blocks (e.g., white, green, blue,etc.). These colors may be generic colors. Alternatively, an electroniccolor palette 622 maybe provided and the viewer can select any colorfrom the palette 622 for the blocks.

In yet another embodiment, the subject invention provides means forautomatically determining a color for one or more blocks based on thecolors of the image from the broadcast program. For example, the meansmay be used to determine a predominant color (and/or texture) of theimage close to the image element being replaced. This predominant colorand/or texture is then used for the optical attributes of the respectivereplacement blocks.

Alternatively the means determines what constitutes the predominantbackground color/texture of the to-be-replaced region by determining,for example, the color of video within that region that is closest tothe color of video surrounding (outside) that region, and therebysetting the color of the to-be-replaced region to the color within theregion that is closest to that of the surrounding video. FIG. 4 billustrates that the picture regions of the video or TV screen that maybe otherwise blacked out with a rounded-corner and a borderconfiguration (see FIG. 4 a) can instead be filled with the closestcolor from the surrounding regions. For example, in the image shown inFIG. 4 b the ship is surrounded with light blue sky and riding in darkblue water. Therefore the upper portion of the frame and the two cornerblocks or triangles are light blue, because the predominant colornearest to (but outside of) the picture area to be filled in is lightblue. Similarly the lower portion of the frame and the corner trianglesor blocks are colored dark blue matching the water. If the picturecontains other elements with different colors at or near the borders,the borders and the corner blocks are colored accordingly. The “nearby”picture region from which the predominant fill color is determined maybe selected, for example, as a strip of the picture bordering the to befilled in region of the picture. For example in FIG. 4 a, a diagonalstrip 160 of the picture adjoining the hypotenuse of the triangularregion 110 may be examined, from which it may be determined that thepredominant color in that diagonal strip is white; white may then beused to fill in region 110. The size of such a nearby bordering picturearea may be determined based upon experience; for example a relativelynarrow region (e.g., 160) whose length is equal to the length of theadjoining to be filled in region (e.g., 110) may be selected.

The predominant color of a region of an image can be detected bycounting the number of pixels in the region with particular pixelvalues. The value of a pixel in a color picture can be represented asthree parameters r, g, b representing the red, green, and bluebrightness of the pixel respectively. The region can, for example, be arectangular region defined by the coordinates of its lower-left cornerand by its width and height. To determine if a given region in a pictureconsists substantially of pixels of only one color, the pixels in theregion are tallied based on their pixel values, as illustrated in theflow chart of FIG. 7. The region is assumed to have its lower leftcorner at coordinates x and y, with a width of i pixels and a height ofj pixels. The process is initialized in steps 702-714. In step 716 theparameter values (that is the parameters r,g,b) of a given pixel areobtained. In performing this computation, it may be desirable to reducethe number of distinct colors recognized so that colors that are verysimilar will be counted in the same tally. This can be achieved byreducing non-overlapping ranges of color to “standard” color values.This can be thought of as a form of quantization and can be achieved inmany ways. One exemplary way is to reduce the precision of each RGBcomponent values to 4 bits, from an initially higher precision. Thisstep is performed in step 718. In steps 720-724 a new pixel is selectedand the process is repeated.

After tallying (step 710), the (preferably quantized) pixel value withthe highest tally is identified in step 726. Steps 728-734 are notexecuted in this case, namely, the case of identifying the predominantcolor of the adjoining picture region.

FIGS. 5 a and 5 b illustrate another example of replacing a portion of abroadcast program where the football scoreboard is placed near the upperleft corner of the image in a block 160. In the present invention, theblock 165 is replaced with a block 170 having similar dimensions, saidblock 170 being filled with a sky-blue video signal to match the skyabove the heads of the players as shown in FIG. 5 b. As the gameprogresses, and the scenes are changed by the director, the color ofblock 170 is changed accordingly in order to implement the embodimentsof FIGS. 4 a, 4 b, 5 a, 5 b, the apparatus 600 includes a scene analyzer624. The scene analyzer 624 determines what automatically what colorshould the blocks be based on one or more of the strategies, and thenfeeds this information into the color selector 620.

In general, various strategies may be used to chose the color of thereplacement blocks. In one embodiment, the system controlling thedisplay device may be able to “copy” the video nearest (e.g.,immediately adjacent) the to-be-replaced region into the to-be replacedregion. For example, the video on each TV signal line nearest to theto-be-replaced video may be copied into the to-be-replaced portion ofeach TV signal line. In this example, there may be some video on TVresolution lines incoming to the display device that is near thehorizontal blanking pulses and is available for copying and free ofobjectionable additions. In some cases, this may be video that is notnormally displayed on the screen. This determination is made by thescene analyzer 624.

In yet other embodiments, the device 600 may replace blocks of videopixels by copying blocks that are known to be in the same position onthe display before and/or after the to-be replaced blocks. For example,by examining MPEG motion vectors, the device may determine that certainblocks of pixels will later move into the to-be-replaced region, and usethose blocks of pixels within that region instead of the pixels thatrepresent the crawl or other distraction. Since MPEG motion vectors mayalso predict motion retrospectively, the display device may alsodetermine that certain blocks of pixels would have emerged from theto-be-replaced region on the display into their current positions. Theseblocks of pixels may then be copied and used to replace theto-be-replaced pixels. If both forward (prospective) and backward(retrospective) motion vectors are available, the display device mayreplace pixels with pixels representing the average of the pixels of theblocks predicted to lead to and result from blocks that would have beendisplayed except for being obscured by some distracting video. Note thatthis particular approach may not succeed if, for example, the motion isonly linear in the horizontal direction, and the obscuring distractionis a horizontal crawl bar.

In yet further embodiments, a “zoom and shift” function may be employedto drive the crawl and/or other distractions off the screen. Today's TVsets may have a zoom function, but typically this does not result indriving all added distractions off the screen, i.e., usually someportions of the top, bottom, or the sides of the original (non-zoomed)picture are still visible and may still be distracting. A “shift”function (using a button or buttons on the remote control 606), maycause the zoomed video to also be shifted up, down, left, and/or rightas required to drive the distracting portion of the zoomed video off thescreen. While the objective of driving all distractions off the screenmight be achieved with one high zoom setting alone, that causes only thecentral portion of the original video picture to be visible. Thus, theside portions of the original video are zoomed off the screenunnecessarily if only one zoom setting capable of driving all possibledistractions on all picture areas off screen is available. Therefore,the “zoom and shift” function may be preferred by the viewers since thisfunction contributes less to video loss. How to implement a zoomfunction is well known to those skilled in the art, as evidenced by itsavailability in TV sets currently on the market. Shifting the pictureposition left or right and/or up or down on the screen may beaccomplished by implementing a variable delay between the video signaland the associated horizontal and/or vertical synchronization (sync)pulses. The incoming video signal may be analyzed by e.g., block overlaydevice 604 and the embedded sync pulses removed. New sync pulses may beinserted into the video signal that are delayed or advanced with respectto the position of the removed sync pulses to cause a shift in theposition of the video on the screen. Note that this process may resultin black picture areas at the picture borders if the shift issufficiently large because the portion of the picture that was at ornear the removed sync pulses (which is black) may become visible. Insome embodiments, these black areas may be filled in with adjoiningcolors in the manner described above.

In some embodiments, heuristics may be used to automatically detect thepresence of a scrolling text display on screen. Such heuristics need notwork perfectly to be of substantial utility and may be based on any or acombination of a variety of characteristics commonly associated with ascrolling text display. Examples of such characteristics include thepresence of a rectangular area near the top or the bottom of the screen,which consists mostly of a foreground color and a background color, or,in embodiments in which motion-compensated video compression isemployed, the presence of a rectangular area near the top or the bottomof the screen in which the motion vectors of pixel blocks in the areahave substantially the same value and point in substantially the samedirection and such a condition persists for at least a predefined periodof time (two seconds for example). The identification can also be doneby recognizing that the video displayed with a constant motion in onedirection overlaid onto a video signal near the bottom edge of thedisplayed image is the unwanted crawl if it includes alpha-numericcharacters. The system can also receive information in the video streamitself that indicates which portions of the stream belong to the videoportion and which belong to the crawl.

Motion search or motion estimation techniques known in the art can beemployed to estimate motion in the portion of the screen in which thepresence of a scrolling text area is to be determined. This techniquecan be implemented regardless of whether or not the video coding employsmotion compensation.

In some embodiments, techniques may be employed to automatically detecta logo. Logos (a.k.a. digital on-screen graphics or “bugs”) are oftenimplemented in one of two ways: as (1) a constant opaque graphicoverlay, or (2) as a constant semi-transparent overlay superimposed onvideo.

In the first case, the presence of a constant opaque graphic overlay maybe detected by detecting blocks of pixels whose values remainsubstantially unchanged over a threshold amount of time in a locationwhere bugs are likely to be found. Preferably, the threshold is chosento be long enough to reduce the probability of false positive, yet shortenough to permit acceptably faster detection.

In the second case, the presence of a semi-transparent overlay may bedetected as an area whose picture values never fall below certainpredetermined threshold over a period of time. During dark scenes orscene changes, when most of the screen is black or dark (i.e. below apredefined threshold), any blocks of “bright” pixels (with brightnessabove a predefined threshold) can be presumed to be part of a logo. Thedetected possible presence of a logo can be combined with pixel valuestatistics over time to arrive at a determination that a logo is indeedpresent.

In some embodiments, automatic logo detection may be limited to certainportions of the screen. For example, the system may apply logo detectiononly to the borders of the picture, or only to a subset of the border(e.g., the lower right corner where logos are often applied bybroadcasters).

One process of detecting a scrolling text display is now described inconjunction with the flow chart of FIG. 7. A substantially two-colorregion in an image can be detected by counting the number of pixels inthe region with particular pixel values. To determine if a given regionin a picture consists substantially of pixels of only two colors, thepixels in the region are tallied based on their pixel values, asillustrated in the flow chart of FIG. 7. The region is assumed to haveits lower left corner at coordinates x and y, with a width of i pixelsand a height of j pixels. The process is initialized in steps 702-714.In step 716 the parameter values (that is the parameters r,g,b) of agiven pixel are obtained. In performing this computation, it may bedesirable to reduce the number of distinct colors recognized so thatcolors that are very similar will be counted in the same tally. This canbe achieved by reducing non-overlapping ranges of color to “standard’color values. This can be thought of as a form of quantization and canbe achieved in many ways. One exemplary way is to reduce the precisionof each RGB component values to 4 bits, from an initially higherprecision. This step is performed in step 718. In steps 720-724 a newpixel is selected and the process is repeated.

After tallying (step 710), the (preferably quantized) pixel values withthe two highest tallies are identified. If the sum of the two highesttallies exceeds a predetermined percentage (e.g. 90%) of the pixels inthe given region, the region is determined to be substantially of twocolors (steps 726-734).

Another process of detecting a scrolling text display is now described.It is well known that a video sequence consists of a sequence ofpictures called frames, each of which is a rectangular array of pixels.Here we adopt the convention that coordinates (0, 0) is at thelower-left corner of the frame. The coordinates of a pixel can bethought of as a position vector, represented as a single symbolnotation-wise.

As discussed above, typically In a color picture, the value of a pixelcan be represented, for example, as three parameters r, g and brepresenting the red, green, and blue brightness values of the pixelrespectively. Just like the position of a pixel, the value of a pixelcan be represented as a single symbol notation-wise.

In the description that follows, a “block” is a rectangular block ofpixel within a frame. For simplicity of exposition, we assume that onlyone predefined block size is used, so that there is no need to specifythe dimensions of a block. As an example, MPEG-2 uses a block size of16×16 pixels for the purpose of motion compensation.

The position of a block can be represented by the position of itslower-left corner.

Let pv₁=(r₁, g₁, b₁) and pv₂=(r₂, g₂, b₂) be two pixel values, a metriccan be defined to measure the “distance” between pv₁ and pv₂. Theexpression |pv₁−pv₂| represents one such a distance measure defined interms of a “Vector norm” function (|·|). Examples of such vector norminclude the standard vector norms. The L-norm of an n-dimensional vectora=(a₁, a₂, . . . , a_(n)) is given by the formula|a|_(L)=(|a₁|^(L)+|a₂|^(L)+. . . +|a_(n)|^(L))^(1/L).

Two exemplary vector norms are the 1-norm and the 2-norm. The 1-norm ofa vector (r, g, b) is the sum |r|+|g|+|b|. The 2-norm of a vector (r, g,b) is the value (r²+g²+b²)^(1/2). Other metric functions for vectors areknown.

From a norm function defined on single pixel values, we can define asimilar function |·| on arrays of pixel values. Let A=[a_(i,j)] be anarray of pixel values. |A| is defined as

${|A| = \sqrt[M]{\left. \underset{i,j}{\Sigma} \middle| a_{i,j} \right|^{M}}},$where |a_(i,j)| is the result of applying a suitable norm function (asexplained before) on the pixel value a_(i,j), and M is a non-negativereal number. Exemplary values of M include 1 and 2.

Let A and B be arrays representing two blocks of pixel values. |A−B|provides a measure of how close A & B are. In particular, if A=B,|A−B|=0.

For the rest of this exposition on motion estimation, we denote by A ablock at position p in the current frame, and B a block at location(p−d) in the previous frame. The motion vector for block A is a value ofd that satisfies some (approximate) minimization criterion. Depending onthe (approximate) minimization criterion used and other implementationdetails, different implementations of motion estimation may not yieldidentical motion vectors, although it is expected that they areidentical or very close most of the time.

The search for the motion vector for a block A is performed in aneighborhood (“search area”) around A. Such a neighborhood can bedefined in terms of maximum (Euclidean) displacement, maximum horizontaland/or vertical displacement. Other ways of defining such an area arepossible.

In some embodiments, the x- and y-components of candidate motion vectorsare limited to integer values. In other embodiments, half-values betweensuccessive integer values are also allowed. It is possible to increasethe granularity of allowable x- and y-component values of candidatemotion vectors even further.

When fractional values are allowed in the x- and y-components of acandidate motion vector, it may be necessary to perform interpolationwhen forming block B at position p−d in the previous frame. Methods forinterpolating pixels, such as linear and cubic interpolations, areknown.

The estimation of motion vector can be performed in many ways. In someembodiments, the motion vector d is chosen from the allowable valueswithin the search area so as to minimize |A−B|. In other embodiments,the motion vector is chosen to reduce |A−B| to below a predeterminedthreshold.

The search for motion vector can be performed in many ways. Astraightforward method is to examine all candidate motion vectors in thesearch area for a value of d that minimizes |A−B|. Another approach isto use hill-climbing to obtain a sequence of displacement vectors thatsuccessively better minimize |A−B|.

If a video sequence contains a region of scrolling content, most of themotion vectors of the blocks in the region will point to the directionof scrolling. Moreover, most of the motion vectors will have (near-)identical magnitudes (reflecting uniform motion in the region). Videosequences may occasionally have short periods of uniform motion in aregion of the frame, even when the region does not generally containscrolling content. To reduce the occurrences of falsely detecting ascrolling region, a time threshold can be employed so that adetermination of probable scrolling region is made only aftercondition(s) associated with a scrolling region persist for some time,say 5 seconds or more.

Any of the conditions below can be treated as indicative of a regionwith scrolling content:

1. The fraction of blocks in the region with motion vectors pointing insubstantially the same direction is ≧a predetermined threshold, forexample 85%.

2. The fraction of blocks in a region with substantially the same motionvector (in terms of both direction and magnitude) is ≧a predeterminedthreshold, for example 85%.

In the case of a news ticker (a.k.a. a “newsticker” or, in TV jargon,“the crawl”) occurring at or near the bottom of a television screen, wemay additionally require the motion vectors to point to the left. Newstickers on different TV channels tend to scroll at a limited range ofspeed. By further requiring the magnitudes of the motion vectors to fallwithin a range typical of news tickers, occurrences of false positivedetection can be further reduced.

One process of detecting an opaque logo is now described in conjunctionwith the flow chart of FIG. 8. The presence of a constant opaque overlaymay be detected by detecting the presence of pixels, in a location wherebugs are likely to be found, whose values remain substantially unchangedover a threshold amount of time. Preferably, the time threshold ischosen to be long enough to reduce the probability of false positive yetshort enough to permit reasonably fast detection.

In this exemplary embodiment, the detection region is defined by thecoordinates of its lower-left corner, and a width and a heightparameter. A 2-dimensional array probable_logo_pixel[ ] stores a flagfor each pixel in the region indicating whether the (quantized) value ofthe pixel has remain unchanged for a threshold amount of time. Theelements of probable_logo_pixel[ ] is continually updated—its content isrefreshed each time a new video frame is available.

In the exemplary embodiment, an optional quantization function isemployed to provide some“noise immunity” i.e. tolerance for (small)fluctuations (over time) in the values of probable logo pixels, whichmay cause false negatives in logo detection. Quantization functions areknown in the art. This is only an example method of providing tolerancefor small, insignificant fluctuations in pixel values. Other knownmethods for the purpose can be substituted.

The exemplary embodiment detects, at an individual pixel level, whethera pixel is part of a probable logo. If the goal of detection is todetect if a probable logo is present, the determination can be made bycounting the number of probable logo pixels in the detection region. Ifthe count is non-zero (or alternatively, above a predefined threshold),a determination of probable presence of a logo is made.

The process is initialized in steps 802-814. In step 816 the parameter pof a current pixel is determined. In step 818 the value of p isoptionally quantized. In step 820 the parameter p is compared to thevalue pv (I,j) and then used to determine whether the pixel associatedwith p likely represents a logo pixel (steps 824, 826) or not (Step822). In steps 828, 830 the next pixel is selected.

One process of detecting a semi-transparent logo is now described inconjunction with the flow chart of FIG. 9. What follows is an adaptationof the previous exemplary embodiment. This adapted embodiment detectsthe probable presence of a semi-transparent logo (or overlay) bydetecting pixels whose brightness value (calculated according to somebrightness measure) does not fall below a predefined threshold, and hasremained so for at least a certain threshold amount of time. The processis initialized in steps 902-914. In step 916 the parameter p of acurrent pixel is determined. The parameter p is compared to a thresholdand then used to determine whether the pixel associated with p likelyrepresents a logo pixel (steps 922, 924) or not (step 920). In steps926, 928 the next pixel is selected.

The heuristics described in the above, and variations thereof, can becombined with other heuristics to improve the accuracy of detection.

In other embodiments, the system may use learning algorithms to detectlogos. To facilitate the robustness of the learning, the system and/orviewer may specify that the learning algorithm should only be applied toa certain region of the picture (i.e., the lower right corner wherelogos are often applied by broadcasters).

In many embodiments, the system may memorize the region of the pictureto blacken, fill in, or otherwise replace with more pleasing pixels on aper-channel and/or per-program basis. For example, if the system detectsor the viewer specifies that a particular logo generally appears in thelower right corner of a certain channel, the system may save adescription of the region to be replaced each time that channel istuned. For even finer functionality, the system may memorize the regionto be replaced on a per-program basis to account for the case wherelogos or other distractions appear at different regions of the videopicture in different programs on the same channel. For example, eventhough the only distraction on the local Fox channel may be a small logoin the lower-right corner of the picture, an annoying scoreboard mayappear in the upper-left corner of the picture during football games onthat channel. In this case, the system may memorize that certain regionsin both the lower-right and upper-left regions of the picture should bereplaced during the football telecasts. The system may, for example,keep track of which programs to replace using a table stored in itsnon-volatile memory, shown in the table below.

Channel Program Region to be Replaced  2 (CBS) Survivor Horizontal pixelX1 to X2; vertical pixel Y1 to Y2 (lower right logo region).  5 (Fox)Football Horizontal pixel X3 to X4; vertical pixel Y3 to Y4 (lower rightlogo region).  5 (Fox) Football Horizontal pixel X5 to X6; verticalpixel Y5 to Y6 (upper left scoreboard region). . . . . . . . . . 32(CNN) (All) Horizontal pixel X7 to X8; vertical pixel Y7 to Y8 (bottomcrawl region).

The above table is only an example. Other variations are possible-tieregion to be replaced may be specified by time offsets (i.e., so manynanoseconds offset from horizontal/vertical sync pulse edges in the caseof analog video signals), rather than by pixel numbers.

In some embodiments, the above-described replacement of annoying videomay be overridden by closed-captioning and/or emergency notificationvideo displayed on top of the replaced video. The option to overridereplacement video may be selectable by the viewer, although preferablythe option to selectively override emergency video should not be madeavailable to the viewer (i.e., emergency video should be “on top of” anyother video, even when the underlying video is the replacement videoaccording to the invention). Likewise, it may be preferable that thedefault condition is that closed-captioned video is on top of the othervideo, but given that consumers do not always understand how toselect/deselect closed captioning, a manufacturer might choose to havereplacement video on top of the closed captioning by default. The choiceof which video (emergency, closed-captioning, replacement, etc.) takesprecedence in terms of layering (i.e., “top layer”, “next layer”,“bottom layer”) may be implemented within the on-screen displaysubsystem of a TV set.

In other embodiments, closed captioning and/or emergency information maybe carried as digital data within the transport streams (MPEG or other),as opposed to analog encoded digital data carried within the verticalblanking intervals (VBIs) of analog television signals. In suchembodiments, if replacement video is selected to overlayclosed-captioned and/or emergency video, the implementation may in factfunction to disable the closed captioning and/or emergency video frombeing generated by the on-screen display subsystem in the firstinstance, as opposed to overlaying it with analog replacement video justbefore the display. This may involve replacing the digital data in theMPEG transport stream or at later points in the MPEG stream decodingprocess with data and/or instructions that cause the closed captioningand/or emergency video not to be generated as analog video for displayin the first instance.

In general, nothing in the above disclosure should be construed to limitthe implementation of replacing portions of video to analog means. Theinvention also foresees implementations that prevent the generation ofannoying video in the first instance, as described above for the case ofclosed captioning and/or emergency information. For example, if logos,scoreboards or the like are specified to be generated by digital datawithin digital video bit streams, then the invention includes the meansto cause the logos, scoreboards, etc. not to be generated by replacingportions of data or instructions in the digital bit streams that wouldotherwise cause them to be generated by a digital receiver.

Although the invention is described in terms of particular embodiments,it is to be understood that the embodiments are merely illustrative ofan application of the principles of the invention. Numerousmodifications may be made and other arrangements may be devised withoutdeparting from the spirit and scope of the invention as defined in theappended claims.

What is claimed is:
 1. A system for modifying video images, said systemcomprising: a processing apparatus receiving a received image having afirst image portion including undesirable content and a second imageportion; a control device in communication with said processingapparatus; and a processing element including a detector for detectingsaid first image portion and an image generator generating a third imageportion having the same size, shape and position as said first imageportion; wherein said processing apparatus, said processing element andsaid control device cooperate to display a processed image composed ofsaid third and said second image portions, whereby the processed imagedoes not include said undesirable content.
 2. The system of claim 1wherein said third image portion is a dynamic image that changes inaccordance with a parameter associated with said received image.
 3. Thesystem of claim 1 wherein said third image portion consists of apreselected image.
 4. The system of claim 1 wherein said first imageportion includes alphanumeric characters.
 5. The system of claim 1wherein said third image portion is a static image.
 6. The system ofclaim 1 wherein said first image portion is one of a text and a logo. 7.The system of claim 1 wherein said first image portion includes ascrolling text.
 8. The system of claim 1 wherein said first imageportion is deleted and replaced by the third portion in said processedimage.
 9. The system of claim 1 further comprising a user interfaceselectively activated by an end user, wherein said control device iscoupled to said user interface and causes said first image portion to bereplaced by said third image portion in response to the activation ofsaid user interface by the user, wherein the end user can select whetherhe wants to see images with said second or third image portion byactivating said user interface.
 10. In a system in which video imagesare transmitted to a receiver as input images, wherein said receiver isassociated with a user interface having a user-activated key and aprocessor, each input image including a first image part withundesirable content, a method of modifying said images comprising:receiving said input images; receiving a command from a user toeliminate said first image part, said command originating from theselection of said user-activated key by the user; determining one of asize, color and position of a replacement image; and responsive to saidcommand, generating a modified image in which said first image part isreplaced with said replacement image and displaying said modified onsaid screen whereby said undesirable content is no longer visible. 11.The method of claim 10 said replacement image is a preselected image.12. The method of claim 11 wherein said command includes at least one ofa user selection of a size, color, and position of said replacementimage.
 13. The method of claim 12 wherein said replacement image isassociated with a parameter that is extrapolated from said input signal.14. The method of claim 10 wherein the presence of said first image partis automatically detected.
 15. The method of claim 10 wherein said firstimage part includes one of a text and a logo.